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ABSTRACT 

As developer and custodian of the General Aptitude 
Test Battery (GATB) , the U.S. Department of Labor (USDOL) is 
responsible for ensuring that its use meets established professional 
standards. Considering the logistical problems of overseeing the use 
of the GATB at more than 2,000 separate sites, the USDOL has 
confronted a monumental task in the face of declining economic 
resources. Use of video administration was recognized as having the 
potential to ease the burdens of test administration and improve test 
reliability because the GATB requires a substantial amount of 
instruction and is complex to time. Video administration tapes were 
developed for English and Spanish versions using the manual section 
on administration and scoring. Separate tapes have been prepared for 
different portions of the GATB. Videotaped administration is used to 
seme degree in 30 states. Overall, responses have been very 
favorable, and the project appears successful. Users report that the 
tapes are particularly useful in administering the dexterity portions 
of the battery. Users' suggestions are being taken into account by 
the USDOL in improving the videos. (SLD) 
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A PROBLEM THAT LED TO DEVELOPING 
VIDEO ADMINISTRATION OF THE GATB FOR TESTING 

AT MULTIPLE SITES 



As developer and custodian of the General Aptitude Test Battery (GATB), the 
U.S. Department of Labor (USDOL) is responsible to ensure that its use at over 2,000 
separate sites meets established professional standards. Such standards are 
contained in Principles for the Validation and Use of Personnel Selection Procedures , 
Society for Industrial and Organizational Psychology, 1987, and Standards for 
Educational and Psychological Testing , American Educational Research Association, 
American Psychological Association, National Council Measurement in Education, 
1 985. Among other things, adherence to these standards ensures reliability of test 
scores used in making personnel selection or individual career decisions. No matter 
how solid the validity evidence of a test whether from rigorous development 
(construct and content) or subsequent use (criterion related), reliability and validity of 
scores can be greatly diminished at any instance of improper use. 

Scope of the Problem and Resources Available 

Considering the logistical problems of overseeing the use of the GATB at more 
than 2,000 separate sites, USDOL does not have sufficient mandated authority or 
funds to exercise close supervision over State Employment Security Agencies (SESAs) 
or affiliated entities authorized to use the GATB. The Wagner Peyser Act of 1933 
established a nationwide system of public employment offices to be administered by 
(SESAs). Before this legislation was amended in 1981, the federal regulations that 
were codified to implement the legislation contained strong language to govern the 
use of tests in screening job applicants and providing vocational guidance to 
individuals. The regulations specifically stated that those who administered the GATB 
and other USDOL tests should be thoroughly trained to provide correct administration 
and interpretation of those tests. A section of the manual that governed Employment 
Service Operations in all SESA local offices provided very specific guidance on the use 
of tests as an extension of the regulations. Funds were allocated for training and 
technical assistance needed to promote adherence to such standards. 

The amendments of 1 981 gave considerable autonomy to SESAs on how they 
would use tests in day-to-day placement and counseling operations. The entire 
Employment Service Manual was rescinded. Funding declined substantially and 
language governing testing was shortened in the new regulations. Passage of the 
amendments coincided with a sharp decline in the number of professional USDOL 
staff designated to do test development research and to provide guidance on the 
effective use of tests developed and maintained by USDOL. 



hough no longer required to take close direction from USDOL, most SESAs prefer and 
also expect to be able to call on USDOL professionals for technical guidance. After 
the amendments were passed USDOL officials still working on the development and 
maintenance of tests, sought solutions on how to operate the program with less 
resources and specific backing in the regulations. One thing that was done was to 
prepare self instruction materials for the use of persons being trained to be GATB test 
administrators. This reduced the amount of direct training needed to develop qualified 
test administrators, but could not make up for the lack of sustained coordination and 
oversight of SESA testing operations from a central USDOL authority. 

Preserving Reliability of Test Scores 

In discussing ways of improving the reliability of tests, Ghiselli (1 964) refers to 
what he calls good "housekeeping" procedures. Among other things, he suggests 
avoiding situations that cause variation in scores that can be attributed to differences 
in environment across testing situations. Some of the factors that operate to cause 
such variation include inconsistencies in adequacy of lighting, ventilation, and freedom 
from distractions in the examination room; and degree of clarity and consistency in 
the instructions given to examinees before and during the administration of the test. 
The purpose of this presentation is to suggest a way of lessening the variation in the 
degree of clarity and consistency of instructions to examinees. It is especially 
intended for tests that are used on numerous occasions and at a large number of 
separate testing sites. 

A possible Solution as the Problem Grew 

As a result of substantial theoretical work that had been done to demonstrate 
the general validity of the GATB across a broad spectrum of work situations, SESAs 
chose in the early to mid 1980s to make considerably greater use of the GATB in 
doing applicant screening and referral . This amplified the problem of monitoring the 
GATB across multiple sites. Though not viewed as something that would completely 
standardize GATB administration across all testing sites, use of video administration 
was viewed as having the potential to do a significant part of the job. It partially 
implemented the suggestion of Ghiselli (1964) to improve reliability by simultaneous 
testing of all individuals. 

Accordingly USDOL in its continuing effort to provide technical assistance to 
SESAs undertook a project to develop video tapes with which to administer the GATB. 
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The project was completed by the Western Test Development Field Center under the 
direction of USDOL program coordinators. The resulting tapes were sxpected to serve 
the following purposes: 

1. Aid responsible staff in administering the GATB. 

2. Standardize the delivery of verbal instructions used in 
administering the GATB. 

3. Accurately time the different short tests that make up the total 
battery that is the GATB. 

4. Reduce time used in delivering instructions. 

5. Reduce potential for tension and confusion being imparted to 
examinees by occasionally flustered test administrators. 

6. Allow test administrators to pay closer attention to examinee 
performance. 

7. Combine video pictures of test form completion and dexterity 
exercises with verbal instructions. 

At this point we need to examine the features of the GATB that make scores 
particularly vulnerable to irrelevant variation when the battery is not carefully 
administered. The GATB is composed of 12 separate parts some of which are further 
composed of subparts or separate exercises. The total working time for all parts of 
the Battery is approximately 48 minutes, but with instructions given and time allowed 
for practice exercises that preceie each part or subpart, the time to administer the 
GATB averages close to 2 1/2 hours. 

Seven of the 12 parts consist of multiple-choice paper-and-pencil tests. Each of 
these parts require specific instructions and an untimed period for practice exercises. 
The actual timed tests for these parts of the Battery allow working time that ranges 
from 5 to 7 minutes. The psychomotor or dexterity portion of the battery consists of 
5 separate parts that each require verbal instructions and an untimed practice period. 
In total the actual tests contain 14 separately timed exercises for which examinees 
are allowed a total of 7 minutes 35 seconds. Actual working time allowed on the 
separately timed exercises ranges from 10 to 90 seconds. 

The point of this description of GATB characteristics is that the administrator 
has a lot of things to remember and a lot to do if instructions are given clearly and 
concisely, and the 21 different testing intervals are timed accurately. A large body 
of evidence shows the GATB to be reliable and valid for the purposes of screening job 
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applicants and providing individual vocational guidance. However, as stated before, 
a let down in any given situation could seriously erode the validity of scores 
associated with that situation. 

The GATB video administration tapes for English and Spanish were developed 
using Section I: Administration and Scoring of the Manual for the USES (jgngral 
Aptitude Test Batte rv as the basis for the script of the video. The tapes simulate in- 
person administration of the GATB by a thoroughly trained and capable administrator. 
The tapes run at a constant, real-time rate of speed for uninterrupted test 
administration, or can be paused as needed during instructions to allow questions 
from examinees. Instructions to "start" and "stop" from the administrator shown in 
the video tape cause a precise timing of the 21 separate test intervals because the 
tape is running in real time. 

Staff with considerable experience administering and interpreting the GATB 
together with academic training and on-the-job experience in audio and video 
communication produced the tapes. This included script preparation, acting, camera 
work, and editing. Separate tapes have been prepared for different forms of the 
GATB, including the Spanish language versions called the BEAG. These tapes are 
available through a supply catalog to all authorized users. 

Now we will show you portions of the tape. 
(Show about 10 minutes of the Dexterity portions in both English and Spanish) 



Experience with the Video 



Judging from the fact that a large number of orders for the tapes have been 
filled and that orders come in daily, one might conclude that the project has been 
successful. Recent discussions with users nationwide confirmed that judgment and 
provided useful suggestions for improvements. Video-taped administration is used to 
some extent in 30 states across the nation. Some states use it in all local 
Employment Service offices. Other states limit its use to the larger local offices. 

Users report that the video is particularly helpful in administering the dexterity 
portions of the battery. Some say that the examinees seem to grasp the irstructions 
more quickly and ask fewer questions. In fact, as answers to very general questions 
users report specific positive information back regarding all of the purposes for which 
the project was undertaken. 



-4- 



One suggestion that came independently from several sources was that the 
actor giving instructions in the video should be as attractive and dynamic as possible 
so as to impart the most possible motivation to examinees. Some observers have gone 
so far as to suggest that the video administrator should wear more fashionable 
clothing. Current versions were deliberately designed to emphasize clarity and 
consistency and to consciously avoid anything that might cause a distraction. In fact, 
were the administrator overly attractive or distinctly unattractive, it might cause at 
least some examinees to be distracted from concentrating on the task at hand. 
Producers of the tapes relied on the assumption that a desire to do well on the 
different tests of the battery would overshadow any interest in critiquing the 
appearance of the person acting as the universal test administrator. Genuine 
enthusiasm coupled with pleasant appearance and the ability to deliver the 
instructions in a natural conversational style are believed to be the qualities most 
important in selecting a universal test administrator. 

One observation that is borne out by suggestions from users across the nation, 
is that the audio and video quality of the tapes needs to be high to start with and that 
the tapes need be monitored very closely when they are used extensively. On 
observing the tape a few weeks ago, WTDFC staff noticed that while the audio 
portion was distinct enough to be serviceable, the instruction to examinees to "stop" 
at the end of most timed intervals was not as loud and clear as it should have been. 
Fortunately the live administrator had noticed this and by independently setting his 
stop watch was able to augment the audio instruction on the tape with his own clear 
command to *'stop." 

The project to produce video tapes for use in GATB administration seems to 
have been successful. All states that use video administration reported bee pleased 
with the results. The tapes should bo considered as a useful tool for use by trained 
administrators. No empirical data is available to document the overall gain in score 
reliability standardized administration through video might bring. Reason alone 
suggests gains worth the effort. 
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